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47 1 distribution MowRatien, and a recognition/ judgment' computer 

2 30 for detecting three-dimensional positions of roads and solid 

3 objects at high speeds based on the distance information inputted 
from the image processor 20 , for identifying a preceding vehicle 
or an obstacle based on the result of the detection and for judging 

6 whether or not an alarm should be issued to avoid a collision 

7 with the preceding vehicle or the obstacle. 

8 The recognition/judgment computer 30 is connected with 

9 sensors such as a vehicle speed sensor 4, a steering angle sensor 

10 5 and the like in order to detect a present traveling condition 

11 of the vehicle and also it is connected with a display 9 provided 

12 at the front of a vehicle driver for informing hazard. Further. 

Q 13 the computer 3 0 is connected with an external interface for 

□ 

fy 14 example for controlling actuators (not shown) which operate so 

35 

83 15 as automatically to avoid a collision with the obstacle or the 

C§ 

fy 16 vehicle traveling ahead. 



M 17 The stereoscopic optical system 10 is composed of a 

nj 18 pair of left and right CCD (Charge Coupled Device) cameras 10a # 

q 19 10b. A pair of stereoscopic images taken by the CCD cameras 10a, 

20 10b are processed in the image processor 20 according to the 

21 principle of triangulation to obtain three-dimensional distance 

22 distribution over an entire image. 
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The recognition/judgment computer 30 reads the C-) 



24 distance distribution information from the image processor 20 
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to detect three-dimensional positions with respect to the 

26 configuration of roads and solid objects such as vehicles and 

27 obstacles at high speeds and judges a possibility of collision 
or contact with these detected objects based on the traveling 
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'•mJ- 1 condition detent- 



condition det ggl ed- te y the vehicle speed sensor 4 and the steering 

2 angle sensor 5 of the self vehicle to inform the vehicle driver 

3 of the result of the judgment through the display 9. 

4 Pig. 2 shows a constitution of the image processor 20 

5 and the recognition/ judgment computer 30. The image processor 

6 20 comprises a distance detecting circuit 20a for producing 

7 distance distribution information and a distance image memory 

8 20b for memorizing this distance distribution information. More 

9 specifically, the distance detecting circuit 20a calculates a 

10 distance to a given object by selecting a small region imaging 

11 an identical portion of the object from the left and right 

12 stereoscopic images taken by the CCD cameras 10a, 10b, 

i — 

W 13 respectively and then obtaining a deviation between these two 

W 14 small regions and outputs in the form of three-dimensional 

S3 15 distance distribution information. 

"J. IS Fig. 9 shows an example of either of images taken by 

M* 17 the left and right CCD cameras 10a # 10b. When this image is 

ru 

fU 18 processed by the distance detecting circuit 20a, the distance 



3 19 distribution information outputted from the distance detecting 

u 

20 circuit 20a is expressed as a distance image as shown in Fig. 

21 10. 

22 The example of the distance image shown in Fig. 10 has 

23 a picture size composed of 600 (laterally) x 200 (longitudinally) 

24 picture elements. The distance data are included in white dotted 

25 portions that correspond to the portions having a large difference 

26 of brightness between two adjacent picture elements aligned in 

27 the left and right direction respectively in the image shown in 

28 Fig. 9. Further, in this example, the distance detecting circuit 



0\J w 1 20a treats^ lie 



distance image as an .image compos-ed of 150 

2 (laterally) x 50 (longitudinally) blocks, i.e., 4x4 picture 

3 elements for one block or one small region. The calculation of 

4 distance is performed for each block of the left and right images. 

5 The recognition/judgment computer 30 comprises a 

6 microprocessor 3 0a primarily for detecting the road configuration, 

7 a microprocessor 30b primarily for detecting solid objects based 

8 on the configuration of a road detected and a microprocessor 30c 

9 primarily for identifying a preceding vehicle or an obstacle based 

10 on the positional information of the detected solid objects and 

11 for judging a possibility of collision or contact with the 

12 preceding vehicle or the obstacle and these microprocessors 30a f 
Q 13 30b, 30c are connected in parallel with each other through a system 
«i 14 bus 31. 

5j 15 The system bus 31 is connected with an interface circuit 

rtj 16 32 to which the distance image is inputted from the distance image 

L 17 memory 20b, a ROM 33 for storing a control program, a RAM 34 for 

18 memorizing miscellaneous parameters produced during 

19 calculations, an output memory 35 for memorizing the result of 



ru 

Q 
Q 

s ~ 20 processing, a display controller 30d for controlling the display 



21 9 and an I/O interface circuit 37 to which signals are inputted 

22 from the vehicle speed sensor 4 and the steering angle sensor 

23 5. 

24 As shown in Pig. 9, the distance image has a coordinate 

25 system composed of a lateral axis i, a longitudinal axis j and 

26 a vertical axis dp with an origin of the coordinates placed at 

27 the left below corner of the distance image. The vertical axis 

28 dp indicates a distance to an object which corresponds to the 
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'UlP* 1 final &^ri*^^tve program goes from S12^^> S130 where it ia 

2 investigated whether or not the process reaches the final group. 

3 When the process does not yet reach the final group, the data 

4 of the next group are read and hereinafter the same processes 

5 are carried out repeatedly, when the process reaches the final 

6 group, the division' of the groups is completed and the program 

7 goes from S130 to S132. 

8 The following steps S132 to S137 are of processes in 

9 which further classifications of "side wall" or "object* are 

10 carried out to raise the accuracy of the classification performed 

11 at S127. After the data of the first group are read at S132, at 

12 S133 approximate straight lines are obtained from the positions 

H 13 (Xi, Zi) within the group according to the Hough transformation 

O 

O 14 or the linear square method to calculate a gradient overall the 

02 15 group. 

Tt 3 ! 
■ W 

ffi IS Then, the program goes to S134 where the group is 

nj 

e 17 reorganized such that the group having a gradient inclined toward 

H- 

pj 18 X-axis is classified into the "object" group and the group having 

PJ 

p 19 a gradient inclined toward Z-axis is classified into the "Bide 

O 

y; 20 wall" group. Further, at S135, miscellaneous parameters of the 

21 group are calculated. With respect to the group classified 

22 "object", these parameters include an average distance which is 

23 calculated from the distance data within the group, X-coordinates 

24 at the left and right ends of the group and the like and with 

25 respect to the group classified "side wall", those parameters 

26 include an arrangement direction of the data (gradient with 

27 respect to Z-axis), z, X coordinates of the front and rear ends 

28 of the group and the like. In this embodiment, in order to raise 
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^ 1 position amJWie- X- coordinates thereof are determined according 

2 to the procedure which will be described hereinafter . 

3 At S202, a node N« corresponding to an end point on the 

4 vehicle side of the selected side wall group is established based 

5 on the 2 -coordinate of the end point and the X- coordinate of the 

6 node N a is established being adjusted to the X-coordinate of the 

7 end point. Next, the program goes to S203 where the next node 

8 N ft *i is established in the direction of the gradient of the side 

9 wall group. Next, when the node N a *i (i^2) is determined, its 

10 direction is established along a direction of the second previous 

11 node. 

12 Then, the program goes to S204 where, as shown in Fig. 
p 13 15, the position of the wall surface is searched by a so-called 
~j 14 "pattern matching"within a specified searching range to extract 

S 15 a solid object Pi for every strip within the searching range. For 

3 

16 example, the searching range in the X-axis direction has i 3 to 

i, 17 5 meters in the X-axis direction and ± 1 meter in the Y-axis 

Jy 18 direction with its center placed at a coordinate {X n8 *i, Z as *i)of 

gl9 the node N»*i established at S203. 

O. 

^20 The matching of the wall surface pattern is performed 

21 to the solid object Pi within the searching range. Fig. 16 shows 

22 an example of the wall surface pattern (weight coefficient 

23 pattern) used for the pattern matching. The wall surface pattern 

24 shown in Fig. 16 is a pattern for the wall surface on the left 

25 side and a symmetric pattern to this pattern is used for the wall 

26 surface on the right side. The lateral axis of this wall surface 

27 pattern coincides with the distance in the X-axis direction and 

28 the longitudinal axis indicates a weight coefficient. A maximum 
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1 on the right-s«e. In the example shown in Fig. 14', the wall surface 
from the 9th node to the 26th node is detected on the right side 
of the self vehicle and the 9th node is denoted as the start point 
N. and the 26th node is denoted as the end point N 0 . These nodes 
are used for later processes as effective nodes. 

Thus processed position of the wall surface is further 
corrected by a program shown in Fig. 7 and Pig. 8 using new data 
obtained from programs shown in Pig. 3 through Pig. 5. 

The programs shown in Fig. 7 and Fig. 8 is a program 
for correcting the wall surface. At S301, it is investigated 
whether or not the start point N fl of the effective nodes is larger 
than the first node N x of the wall surface model. When N a ■ N*, 
the wall surface has been already detected up to the first node 
Ni, the program skips to S306. When N« > Ni, the program goes to 
S302 where the previous node N..i (i -1,2,3 etc.) is established. 
Then, at S303 the wall surface pattern is searched and at S304 
the X- coordinate of the wall surface is determined based on the 
result of searching. 

Next, the program goes from S304 to S305 where it is 
investigated whether or not the process has reached the first 
node. If not yet reached the first node Ni, the steps S302 to S304 
are repeated to continue the searching of the wall surface 
position up to the node Hi. When the processes up to the first 
node Ni are finished, the program goes to S306 where it is checked 
whether or not the end point N c of the effective nodes is smaller 
than the last node N ao of the wall surface model (for example, 
node N«i in case of the wall surface model constituted of 41 nodes) . 

As a result of this, when N c » N BO i that is, the wall 
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^1 surface has £S»-already detected up to the last node, the program 

2 skips from S306 to S311. When N € < N 8 o, the program goes from S306 

3 to S307 where the node N Q .i after the end point N* is successively 

4 established and further at S308 the pattern matching of the wall 

5 surface is performed. According to the result of the pattern 

6 matching, at S309 the X-coordinate of the wall surface is 

7 determined and then at S310 it is checked whether or not the process 

8 . has reached the last node N B « . The matching of the wall surface 

9 position is continued until the last node N B « and when the processes 

10 up to the last N ae is finished, the program goes to S311. 

11 These processes of establishing the nodes, the matching 

12 of the wall surface pattern and the determination of the X- 

q 13 coordinate at the steps S302 to S304 and the steps S307 to S309, 

f*v 

pi 14 are the same as the processes at the steps S203, 204 and S205 

H 15 in the aforementioned program of the wall surface detecting 

JT. 16 process. 

iy 

17 The processes after S311 are for correcting the 

18 position (X-coordinate) of respective nodes from the first node 
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19 Ni to the last node N ea , First, at S311 the data of the first node 

O 

20 Ni is set and the program goes to S312. The processes from S312 

21 to S321 are repeatedly carried out by successively setting the 

22 data of the next node. 

23 At S312, the wall surface at the node Ni is searched 

24 and at S313 it is checked whether or not the wall surface is 

25 detected by the pattern matching. If it is judged that the wall 

26 surface is detected, the program goes from S313 to S314 where 

27 it is investigated whether or not the difference between the 

28 position X P v of the wall surface and the position X n i of the node 
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